Skip to content

test: run the suite in CI and cover the 0.5.0 fixes - #280

Merged
mkb79 merged 1 commit into
masterfrom
ci/add-test-suite
Aug 7, 2026
Merged

test: run the suite in CI and cover the 0.5.0 fixes#280
mkb79 merged 1 commit into
masterfrom
ci/add-test-suite

Conversation

@mkb79

@mkb79 mkb79 commented Aug 7, 2026

Copy link
Copy Markdown
Owner

The repository had pytest, coverage and Ruff configured in pyproject.toml but nothing that ran them, no pytest dependency, and a root test.py whose entire body was print("fake test"). The one real test — tests/test_cmd_manage.py, contributed with #269 — has never been executed anywhere.

Infrastructure

  • pytest added to the dev group, testpaths narrowed to tests, the placeholder deleted
  • New tests.yml: runs on pull requests and pushes to master, across Python 3.11 – 3.14, with permissions: contents: read, a concurrency group that cancels superseded runs, and fail-fast: false so one version's failure does not hide the others
  • uv sync --locked doubles as the lockfile check the pull request workflows otherwise lack — a pyproject.toml change without a matching uv.lock now fails here

Ruff is deliberately not gated. It reports 431 findings on the existing code, so a blocking job would fail on arrival. Cleaning that up is its own piece of work. tests/ itself is Ruff-clean.

Tests

56 tests, 0.3s, covering the bugs fixed for 0.5.0:

File Covers
test_utils_datetime both API timestamp shapes, offsets other than Z, rejection of ambiguous values, UTC normalization of the --start-date/--end-date type
test_models_dates the date filter across both shapes and naive and aware bounds, items with no usable date, is_published including the AudioPart parent fallback
test_download_queue the deadlock — failing jobs must not strand the queue, the abort must stop queued work while letting running downloads finish
test_download_cli the command itself must raise on a failed job
test_exceptions ItemNotPublished must not fail while building its own message

Why these tests are worth having

Green tests prove nothing on their own, so each fix was reverted in turn to check the suite goes red:

Fix reverted Result
parse_api_datetime back to a single format (#264) 15 failed
library_status guard removed (#268) 6 failed
AudioPart publication fallback removed (#268) 4 failed
ItemNotPublished catch narrowed (#268) 4 failed
run.raise_for_errors() call deleted (#256) 2 failed
the original pre-#271 consume() restored (#235) 6 failed, and the run took 25s instead of 0.3s — those are the timeouts firing on the deadlock
nothing reverted 56 passed

That last row is the point of test_download_cli: testing raise_for_errors() in isolation would have stayed green if the command stopped calling it, so the wiring is exercised through the real cli with only the network edges mocked.

Not covered

The download paths that only log a failure rather than raising — an unknown ASIN, a cover without a URL, a non-success Status from the downloader. Those are the remaining half of #256 and need the fix before they can be tested.

The repository had pytest, coverage and Ruff configured in pyproject.toml
but nothing that ran them, no pytest dependency, and a root `test.py` whose
entire body was `print("fake test")`. The one real test, added with #269,
has never been executed anywhere.

Add pytest to the dev group, narrow `testpaths` to `tests`, drop the
placeholder, and run the suite on every pull request and on pushes to master
across Python 3.11 to 3.14. `uv sync --locked` doubles as the lockfile check
the pull request workflows otherwise lack.

Ruff is deliberately not gated. It currently reports 431 findings on the
existing code, so a blocking job would fail on arrival. Cleaning that up is
its own piece of work.

The tests themselves cover the bugs fixed for 0.5.0, each written so that it
fails again if the fix is reverted:

  test_utils_datetime    both API timestamp shapes, offsets other than Z,
                         rejection of ambiguous values, UTC normalization
                         of the --start-date/--end-date option type
  test_models_dates      the date filter across both shapes and naive and
                         aware bounds, items carrying no usable date at all,
                         is_published including the AudioPart parent
                         fallback
  test_download_queue    the deadlock: failing jobs must not strand the
                         queue, the abort must stop queued work while
                         letting running downloads finish, and a run that
                         saw failures must raise
  test_download_cli      the command itself has to raise on a failed job;
                         testing raise_for_errors() alone would stay green if
                         the call were dropped from the command
  test_exceptions        ItemNotPublished must not fail while building its
                         own message
@mkb79
mkb79 merged commit 5c29cdc into master Aug 7, 2026
5 checks passed
@mkb79
mkb79 deleted the ci/add-test-suite branch August 7, 2026 16:36
mkb79 added a commit that referenced this pull request Aug 7, 2026
Ruff has been configured in pyproject.toml for a while but never ran
anywhere. Turning it on as-is would fail immediately: 442 findings in 29
files, and 29 files the formatter has never touched.

Both jobs run over the whole repository rather than a list of directories,
so `pyi_entrypoint.py`, which the release build uses, `utils/` and the docs
config are covered too, and a new top-level file cannot slip past.

Rather than switch the rules off globally, pin what each file already breaks.
A global ignore list would be an amnesty: a new violation of the same rule
would then pass anywhere, including in files written from scratch. With
`per-file-ignores` a new file, and a rule a file does not already break, both
still fail.

The one thing this does not catch is another violation of a rule already
listed for that same file — `per-file-ignores` keys on file and rule, not on
individual findings. Ruff has no native baseline for that yet.

`ruff format` is gated the same way, through `[tool.ruff.format].exclude`.
That leaves the 25 legacy files alone while every new file is checked from
the start, and avoids a 1400-line reformat that would collide with the eight
external pull requests still open. The three test files added in #280 were
not in that state for long, so they are formatted here instead of listed.

Ruff itself is pinned. The enabled rule families pull in new stable rules on
an upgrade, which would turn the gate red without anyone touching the code.

Both run as separate jobs so a failure names which one, without the Python
matrix the tests need.

Shrinking this is the point: fix a rule in a file, drop its code from the
list, or run `ruff format` on one file and remove it from the exclude list.
197 of the findings are safe autofixes, which is the obvious next step.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant